PATENT COOPERATION TREATY 



PCT/G BOO/00242 



From the INTERNATIONAL BUREAU 



PCT 

NOTIFICATION OF ELECTION 

(PCT Rule 61.2) 


To: 

Commissioner 

US Department of Commerce 
United States Patent and Trademark 
Office, PCT 

2011 South Clark Place Room 
CP2/5C24 

Arlinaton VA OOOdo 
ETATS-UNIS D'AMERIQUE 

in its capacity as elected Office 


Date of mailing (day/month/year) 

06 November 2000 (06. 1 1 .00) 


International application No. 
PCT/G BOO/00242 


Applicant's or agenfs file reference 
30990001 WO 


International filing date (day/month/yea r) 
28 January 2000 (28.01.00) 


Priority date (day/month/year) 

12 February 1999 (12.02.99) 


Applicant — . 

FROHLICH, David, Mark et al 



The designated Office is hereby notified of its election made: 

[X] in the demand filed with the International Preliminary Examining Authority on: 

04 September 2000 (04.09.00 ) 

Q in a notice effecting later election filed with the International Bureau on: 



2. The election |"x] was 

□ 



was not 



The International Bureau of WIPO 


Authorized officer 




34, chemin des Colombettes 


Olivia TEFY 




1211 Geneva 20, Switzerland 




Facsimile No.: (41-22) 740.14.35 


Telephone No.: (41-22) 338.83.38 




r-orm PCT/IB/331 (July 1992) 1 

GB0000242 



PCT 

REQUEST 



The undersigned requests that the present 

international application be processed 
according to the Patent Cooperation Treaty. 



. For receiving Office use only 



International Application No. 



International Filing Date 



Name of receiving Office and "PCT International Application" 



Applicant's or agent's file reference 

(if desired) (12 characters maximum) 3099000 1 WO 



of residence is indicated below J 

Hewlett-Packard Company 
3000 Hanover Street 
Palo Alto 
CA 94304 
US 



State (that is, country) of nationality: 
US 


State (that is. country) of 

US 


residence: 




Bos No. Ill FURTHER APPLICANTS) AND/OR (FURTHER) INVENTOR(S) 





Box No. I TITLE OF INVENTION 

Digital Camera With Sound Recording 



Box No. II APPLICANT 



e « numc, for a legal entity, full official 
and name of counby. The country of the 
e (that is, country) of residence if no State 



[ | This person is also inventor. 



Telephone No. 



Facsimile No. 



Teleprinter No. 




of residence is indicated below.) 



FROHL1CH, David Mark 
17 Briar Wood 
Westbury-on-Trym 
Bristol BS9 3SS 
GB 



zapp 



\try) of residence if n 



State (that is, country) of nationality: 
GB 



This person is: 

| | applicant only 

|^| applicant and inventor 

I I inventor only (If this check-box 
' 1 is marked, do not fill in below.) 



State (that is, country) of residence: 
GB 



This person is applicant I I 
for the purposes of: I— J 



all designated 
States 



Further applicants and/or (further) inventors are indicated on a continuation sheet. 



□ all designated States except HTl the United States [ I the abdicate dm 
the United States of America L*J of America only I 1 the Supplemental Box 



Box No. IV AGENT OR COMMON REPRESENTATIVE; OR ADDRESS FOR CORRESPONDENCE 



The person identified below is hereby/has been appointed to act on behalf 
of the applicant(s) before the competent International Authorities as: 



|y| agent [ | common representative 



Name and address: (Family name followed by given name; for a legal entity. M^ffi^al 
name anu *iuui» s Agnation. The address must include postal code and name of country.) 



designatii 

LAWRENCE, Richard Anthony 
Hewlett-Packard Limited 
Intellectual Property Section 
Filton Road 

Stoke Gifford, Bristol BS34 8QZ 
GB 



Telephone No. 

(0)117-312-8295 



Facsimile No. 

(0)117-312-8941 



Teleprinter No. 



I—I Address for correspondence: Mark this check-box where no agent or common representative is/has been appointed and the 
I I space above is ..sed'instead to indicate a special address to whicH correspondence should be sent. 



space above is used i 

Form PCT/RO/101 (first sheet) (Jury 1998; reprint January 2000) 



See Notes to the request form 



) 



Sheet No. f. 

ConHnuationorBoxNo.111 FURTHER APPLICANTS AND/OR (FURTHER) INVENTORS 



If none of the following sub-boxes is used, this sheet should not be included in the request. 



HICKEY, Marianne 
40 Maple Road 
Bristol BS7 8RQ 
GB 



This person is: 

| | applicant only 

jy| applicant and inventor 

□ inventor only (If this check-box 
is marked, do not fill in below.) 



State (that is, country) of nationality: 



GB 



State (that is, country) of residence: 



GB 



This person is applicant 
for the purposes of: 



j | all designated 



| States 



□ all designated States except 
the United States of America 



rCTi the United States 
|J» 1 of America only 



□ 



the States indicated in 
the Supplemental Box 



Name and address: (Family nameJbUoved by ^jg^SlJ^^^^S^SffSSi 

The address must include postal code and name of country. The country oj we am !^£"£"jj\Y' Jr 
^xfthVap^canT^ Stale (that is, country) of residence if no State of residence is indicated below.) 



This person is: 

j | applicant only 

| | applicant and inventor 

□ inventor only (If this check box 
is marked, do not fill in below.) 



State (that is, country) of nationality: 



State (that is, country) of residence: 



This person is: 



[ j applicant only 

[ | applicant and inventor 

□ inventor only (If this check-box 
is marked; do not fill in below.) 



State (that is, country) of nationality: 



State (that is. country) of residence: 



This person is applicant 
for the purposes of: 



□ all designated I 1 all designated States except 
States I 1 *e United States of America 



□ the United States I 1 the States indicated in 
of America only 



□ the States indicated in 
the Supplemental Box 



This person is: 

| | applicant only 

| | applicant and inventor 

I I inventor only (If this check-box 
1 1 is marked, do not fill in below.) 



State (that is, country) of nationality: 



State (that is, country) of residence: 



TOspersonisapplieant pi [H &$3Ss^a£SS. □ 3? AnSW □ «hS sgg5«£l b5 

for the purposes of: I— i ° 1 1 _^ ==== __ ====== =============== 



□ Further applicants and/or (further) inventorsare i ndicated on another continuation sheet. 
Form PCT/RO/101 (continuation sheet) (July 1998; reprint July 1999) 



See Notes to the request form 



SJieet No. 3. (| . 



Box No.V DESIGNATION OF STATES 



□ EA 



S EP 



□ OA 



The following designations are hereby made under Rule 4.9(a) (mark the applicable check-boxes; at least one must be marked): 
Regional Patent 

n AP ARIPO Patent: GH Ghana, GM Gambia, KE Kenya, LS Lesotho, MW Malawi, SD ^Sudan, SL Sierra Leone SZ Swaziland, 
U TZUnitc2 I Rcoubhfof Tanzania, UG Uganda, ZW Zimbabwe, and any other State which is a Contracting State of the Harare 
Protocol and of the PCT L1 . _ w 

Eurasian Patent: AM Armenia, AZ Azerbaijan, BY Belarus, KG ^yT^zstan, KZ 1^4™, ^^ b ^^ffiS{ 
RU Russian Federation, TJ Tajikistan,TM Turkmenistan, and any other State which is a Contracting State of the Eurasian Fatent 

Convention and of the PCT . mr _ ~ 

Eurooean Patent* AT Austria BE Belgium, CH and LI Switzerland and Liechtenstein, CY Cyprus DE Germany, 
DK Kar^ ES Spain FI Finfand, FR France^, GB United Kingdom, GR Greece, IE Ireland, IT Itaft LU Luxembourg 
MC Nfon^oVNL Portugal, SE Sweden, and any other State which is a Contracting State of the European Patent 

Convention and of the PCT ^ m 
OAPI Patent- BF Burkina Faso, BJ Benin, CF Central African Republic, CG Congo, CI Cote^ d I Ivoirc, CM Cameroon 
GA Gabon GN Guinea GW Guinea-Bissau, MLMali, MR Mauritania, NE Niger SN Senegal, TD Chad, TG Togo >, and any 
XerState which is a member State of OAPI and a Contracting State of the PCT (if other kind of protection or treatment desired, 

specify on dotted line) 

National Patent (if other kind of protection or treatment desired, specify on dotted hne): 

□ AE United Arab Emirates □ LR Liberia 

□ AL Albania □ LS 

□ AM Armenia Q LT 

□ AT 

□ AU 

□ AZ 

□ BA 

□ BB 

□ BG 

□ BR 

□ BY 

□ CA 



Lesotho . . . 
Lithuania 
Luxembourg 
Latvia 

Morocco . . . 



Austria □ LU 

Australia □ LV 

Azerbaijan □ MA 

Bosnia and Herzegovina □ MD Republic of Moldova 

Barbados □ MG Madagascar 

Bulgaria □ MK The former Yugoslav Republic of Macedonia 

Brazil 

Belarus □ MN Mongolia 

Canada □ MW Malawi 



□ CH and LI Switzerland and Liechtenstein □ MX 

□ CN China □ NO 

□ CR Costa Rica □ NZ 

□ CU Cuba □ PL 

□ CZ Czech Republic □ PT 

□ DE Germany □ RO 

□ DK Denmark □ RU 



□ DM Dominica 



□ SD 



□ EE Estonia □ SE 

□ ES Spain □ SG 

□ FI Finland □ SI 

□ GB United Kingdom □ SK 

□ GD Grenada □ SL 

□ GE Georgia □ T J 

□ GH Ghana □ TM 



□ GM Gambia 



□ TR 



□ HR 

□ HU 

□ ID 

□ IL 

□ IN 

□ IS 
GO JP 

□ KE 

□ KG 

□ KP 

□ KR 

□ KZ 

□ LC 

□ LK 



Croatia □ TT 

Hungary D TZ 

Indonesia CD UA 

Israel B VG 

India BUS 

Iceland 

Japan B VZ 

Kenya □ VN 

Kyrgyzstan 

Democratic People's Republic of Korea . . . 



Mexico 

Norway 

New Zealand 

Poland 

Portugal 

Romania 

Russian Federation 

Sudan 

Sweden 

Singapore 

Slovenia 

Slovakia 

Sierra Leone 

Tajikistan 

Turkmenistan 

Turkey 

Trinidad and Tobago 

United Republic of Tanzania 

Ukraine 

Uganda 

United States of America . 



□ YU 

□ ZA 

□ ZW 



Uzbekistan . 
Viet Nam . . 
Yugoslavia . 
South Africa 
Zimbabwe . . 



t> ui 'fY™ Check-boxes reserved for designating States which have 

Republic of Korea become party to the PCT after issuance of this sheet: 



Kazakhstan 
Saint Lucia 
Sri Lanka 



□ 
□ 



Precautionary Designation Statement: In addition to the designations made above the applicant also makes under Rule 4 . 9 (b) all other 
designations which would be permitted under the PCT except any designation(s) indicated m the Supplemental Box as bemg excluded 
fronfthe scope of this statement. The applicant declares that those additional designators are subject to confirmation and that any 
designation which is not confirmed before the expiration of 1 5 months from the priority date is to be regarded as w.thdrawn by the applicant 
at the expiration of that time limit. (Confirmation (inchtdmgfeej) must reach thereceivingOffice within the 15-month ame ImiLJ 



Form PCT/RO/101 (second sheet) (January 2000) 



See Notes to the request form 



Sheet No. 



Box No. VI PRIORITY CLAIM 



| | Further priority claims are indicated in the Supplemental Box. 



Filing date 
of earlier application 
(day/month/year) 



item(l) (12. 02. 99) 

12 February 1999 



item (2) 



item (3) 



Number 
of earlier application 



99301045.3 



Where earlier application is: 



national application: 
country 



regional application:* 
regional Office 



EP 



international application: 
receiving Office 



I The receiving Office is requested to prepare and transmit to the c f^!^°£ 
1 of the earlier application(s) (only if the earlier application was filed wUh *£J&^'££Z£ 
purposes of the present international application is the receiving Office) jdentified above as item(s). 



♦ Where the earlier application is anARIPO application, it is mandatory ta ^^J^^^^S^% 
Convention for the Protection of Industrial Property for which that earlier application was filed (Rule 4. lOfoJOVJ. 



elemental Box at least one country party to the Paris 



See Supplemental Box. 



Boi No. VP INTERNATIONAL SEARCHING AUTHORITY 



Choice of International Searching Authority (ISA) 

{if two or more International Searching Authorities are 
competent to carry out the international search, indicate 
the Authority chosen; the two-letter code may he used) : 

ISA/ EP 



Request to use results of earlier search; reference to that search (if an earlier 
search has been carried out by or requested from the International Searching Authority). 



Date (day/month/year) 

12 February 1999 



Number 
99301045.3 



Country (or regional Office) 

EP 



Box No. Vffl CHECK LIST; LANGUAGE OF FILING 



This international application contains 
the following number of sheets: 

request : 

desenption (excluding 
sequence listing part) 

claims 

abstract 

drawings 

sequence listing part 
of description 



15 
4 
1 
5 



Total number of sheets : 



This international application is accompanied by the itcm(s) marked below: 

1 . 0 fee calculation sheet 

2. 13 separate signed power of attorney 

3 0 copy of general power of attorney, reference number, if any: 
4. □ statement explaining lack of signature 
5 □ priority document(s) identified in Box No. VI as item(s): 

6. □ translation of international application into (language): 

7. □ separate indications concerning deposited microorganism or other biological material 

8. □ nucleotide and/or amino acid sequence listing in computer readable form 

9. 0 other (specify): EP Search Report 



Figure of the drawings which 
should accompany the abstract: 2b 



Language of filing of the . . 
international application: cn 9" bn 



Box No. K SIGNATURE OF APPLICANT OR AGENT 



Me* toeachsignature .irulicate thenar CapOCity * obviOUSfr ° m ^ rVqUeS °- 





Richard Anthony Lawrence 



For receiving Office use only ■ 



1 . Date of actual receipt of the purported 
international application: 



3. Corrected date of actual receipt due to later but 
timely received papers or drawings completing 
the purported international application: 



4. Date of timely receipt of the required 
corrections under PCT Article 1 1(2): 



5. International Searching Authority , ojl / 
(if two or more are competent): / 



□ Transmittal of search copy delayed 
until search fee is paid. 



2. Drawings: 
| [ received: 

\ | not received: 



Date of receipt of the record copy 
by the International Bureau: 



For International Bureau use only ■ 



Form PCT/RO/101 (last sheet) (Jury 1998; reprint January 2000) 



See Notes to the request form 



Tfie demand must be filed directly with the competent International Preliminary Examining Authority or, if two or more Authorities are competent, 
with the one chosen by the applicant The full name or two-letter code of that Authority may be indicated by the applicant on the line below: 

IPEA/ EP 



PCT 



CHAPTER n 



DEMAND 

under Article 3 1 of the Patent Cooperation Treaty: 
The undersigned requests that the international application specified below be the subject of 
international preliminary examination according to the Patent Cooperation Treaty and 
hereby elects all eligible States (except where otherwise indicated). 



Identification of IPEA 



For International Preliminary Examining Authority use only 



Date of receipt of DEMAND 



Box No. I IDENTIFICATION OF THE INTERNATIONAL APPLICATION 



Applicant's or agent's file reference 
30990001 WO 



International application No. 
PCT/GB00/00242 



International filing date (day/month/year) 
28 January 2000 (28/01/00) 



(Earliest) Priority date (day/month/year) 
12 February 1999 (12/02/99) 



Title of invention 



Digital Camera With Sound Recording 



Box No. II APPLICANTS) 



Name and address: (Family name followed by given name; for a legal entity, full official designation. 
The address must include postal code and name of country.) 

Hewlett-Packard Company 
3000 Hanover Street 
Palo Alto 
CA 94304 
US 



Telephone No.: 



Facsimile No.: 



Teleprinter No.: 



State (that is, country) of nationality: 
US 



State (that is, country) of residence: 
US 



Name and address: (Family name followed by given name; for a legal entity, full official designation. The address must include postal code and name of country.) 

FROHLICH, David Mark 
17 Briar Wood 
Westbury-on-Trym 
Bristol BS9 3SS 
GB 



State (that is, country) of nationality: 
GB 



State (that is, country) of residence: 
GB 



Name and address: (Family name followed by given name; for a legal entity, full official designation. The address must include postal code and name of country.) 

HICKEY, Marianne 
40 Maple Road 
Bristol BS7 8RQ 
GB 



State (that is, country) of nationality: 
GB 



State (that is, country) of residence: 
GB 



| | Further applicants are indicated on a continuation sheet. 



Form PCT/IPEA/401 (first sheet) (July 1998; reprint July 1999) 



See Notes to the demand form 



o 

Sheet No. . . 



International application No. 

PCT/GBOO/00242 



Box No. in AGENT OR COMMON REPRESENTATIVE; OR ADDRESS FOR CORRESPONDENCE 



The following person is \)(\ agent | | common representative 

and jX | has been appointed earlier and represents the applicants) also for international preliminary examination. 

| 1 is hereby appointed and any earlier appointment of (an) agent(s)/common representative is hereby revoked. 

| | is hereby appointed, specifically for the procedure before the International Preliminary Examining Authority, in addition to 
the agent(s)/common representative appointed earlier. 



Name and address: {Family name followed by given name; for a legal entity, foil official designation. 
The address must include postal code and name of country.) 

LAWRENCE, Richard Anthony 

Hewlett-Packard Limited 

Intellectual Property Section 

Filton Road 

Stoke Gifford 

Bristol BS34 8QZ 

GB 



Telephone No.: 

(0)117-312-8295 



Facsimile No.: 

(0)117-312-8941 



Teleprinter No.: 



□ Address for correspondence: Mark this check-box where no agent or common representative is/has been appointed and the 
space above is used instead to indicate a special address to which correspondence should be sent. 



Box No. IV BASIS FOR INTERNATIONAL PRELIMINARY EXAMINATION 



Statement concerning amendments:* 

1 . The applicant wishes the international preliminary examination to start on the basis of: 
1X1 the international application as originally filed 

the description I I as originally filed 

I | as amended under Article 34 

the claims | | as originally filed 

| | as amended under Article 1 9 (together with any accompanying statement) 
| | as amended under Article 34 

the drawings I I as originally filed 

| | as amended under Article 34 

2. | | The applicant wishes any amendment to the claims under Article 1 9 to be considered as reversed. 

3 . The applicant wishes the start of the international preliminary examination to be postponed until the expiration of 20 months 
from the priority date unless the International Preliminary Examining Authority receives a copy of any amendments made 
under Article 1 9 or a notice from the applicant that he does not wish to make such amendments (Rule 69. 1 (d)). (This check- 
box may be marked only -where the time limit under Article 19 has not yet expired) 

* Where no check-box is marked, international preliminary examination will start on the basis of the international application 
as originally filed or, where a copy of amendments to the claims under Article 19 and/or amendments of the international application 
under Article 34 are received by the International Preliminary Examining Authority before it has begun to draw up a written opinion 
or the international preliminary examination report, as so amended. 



Language for the purposes of international preliminary examination: .English 

IX | which is the language in which the international application was filed. 

1 | which is the language of a translation furnished for the purposes of international search. 

1 | which is the language of publication of the international application. 

| | which is the language of the translation (to be) furnished for the purposes of international preliminary examination. 



Box No.V ELECTION OF STATES 



The applicant hereby elects all eligible States (that is, all States which have been designated and which are bound by Chapter II of 
the PCI) 

excluding the following States which the applicant wishes not to elect: 



Form PCT/IPE A/401 (second sheet) (July 1998; reprint July 1999) 



See Notes to the demand form 



• 




• 


Sheet No. 




PCT/GBOO/00242 


Box No. VI CHECKLIST 


The demand is accompanied by the following elements, in the language referred to in 
Box No. IV, for the purposes of international preliminary examination: 

1. translation of international application : sheets 




For International Preliminary 
Examining Authority use only 

received not received 

□ □ 


s.. alllCllUJIlCIllo U I 111 CI /VTUUIC J>t . &llGGl»» 




1 — 1 1 — 1 
1 1 1 — 1 


3. copy (or, where required, translation) of 

amendments under Article 19 sheets 




□ □ 


4. copy (or, where required, translation) of 

statement under Article 19 : sheets 




□ □ 


5. letter : sheets 

6. other (Specif) sheets 




□ □ 

□ □ 


The demand is also accompanied by the item(s) marked below: 

1. J^X] fee calculation sheet 4. | | statement explaining lack of signature 

2. | | separate signed power of attorney 5. | I nucleotide and or amino acid sequence listing in 

1 — ■ computer readable form 

3. 1 1 copy of general power of attorney; 

' — 1 reference number, if any: 6. Q other (specify): 


Box No. VII SIGNATURE OF APPLICANT, AGENT OR COMMON REPRESENTATIVE 


Next to each signature, indicate the name of the person signing and the capacity in which the person signs (if wcA capacity is not obvious from reading the demand). 


Richard Anthony Lawrence 











I. 


Date of actual receipt of DEMAND: 


2. 


Adjusted date of receipt of demand due 
to CORRECTIONS under Rule 60.1(b): 


3. 


1 — 1 The date of receipt of the demand is AFTER the expiration of 1 9 months I — I The applicant has been 
1 — 1 from the priority date and item 4 or 5, below, does not apply. " — ' informed accordingly. 


A 1 — 1 The date of receipt of the demand is WITHIN the period of 19 months from the priority date as extended by virtue of 
4 * I 1 Rule 80.5. 


5. 


f~ 1 Although the date of receipt of the demand is after the expiration of 19 months from the priority date, the delay in arrival 
1 — 1 is EXCUSED pursuant to Rule 82. 



For International Bureau use only 



Demand received from IPEA on: 



Form PCT/IPEA/401 (last sheet) (July 1998; reprint July 1999) 



See Notes to the demand form 




PATENT COOPERATION TREATY,, 

PCT 

INTERNATIONAL PRELIMINARY EXAMINATION REPORT 

(PCT Article 36 and Rule 70) 



Applicant's or agent's file reference 
30990001 WO 


. See Notification of Transmittal of International 
FOR FURTHER ACTION Preliminary Examination Report (Form PCT/IPEA/416) 


International application No. 
PCT/GB00/00242 


International filing date (day/month/year) 
28/01/2000 


Priority date (day/month/year) 
12/02/1999 


International Patent Classification (IPC) or nat 
H04N1/21 


ional classification and IPC 


Applicant 

HEWLETT-PACKARD COMPANY et al. 



1. This international preliminary examination report has been prepared by this International Preliminary Examining Authority 
and is transmitted to the applicant according to Article 36. 

2. This REPORT consists of a total of 5 sheets, including this cover sheet. 

□ This report is also accompanied by ANNEXES, i.e. sheets of the description, claims and/or drawings which have 
been amended and are the basis for this report and/or sheets containing rectifications made before this Authority 
(see Rule 70.16 and Section 607 of the Administrative Instructions under the PCT). 

These annexes consist of a total of sheets. 



3. This report contains indications relating to the following items: 



I 




II 


□ 


III 


□ 


IV 


□ 


V 




VI 


□ 


VII 




VIII 


□ 



citations and explanations suporting such statement 

Certain documents cited 

Certain defects in the international application 



Date of submission of the demand 
04/09/2000 



Name and mailing address of the international 
preliminary examining authority: 
European Patent Office 

D-80298 Munich 
Tel. +49 89 2399 - 0 Tx: 523656 epmu d 

Fax: +49 89 2399 - 4465 



Date of completion of this report 



22.11.2000 



Authorized officer 

Moorhouse, D 

Telephone No. +49 89 2399 8631 




Form PCT/IPEA/409 (cover sheet) (January 1994) 



INTERNATIONAL PRELIMINARY 
EXAMINATION REPORT 



International application No. PCT/G BOO/00242 



I. Basis of the report 

1 . This report has been drawn on the basis of (substitute sheets which have been furnished to the receiving Office in 
response to an invitation under Article 14 are referred to in this report as "originally filed" and are not annexed to 
the report since they do not contain amendments (Rules 70. 16 and 70.17).): 
Description, pages: 

1-15 as originally filed 



Claims, No.: 

1-24 

Drawings, sheets: 

1/5-5/5 



as originally filed 



as originally filed 



2. With regard to the language, all the elements marked above were available or furnished to this Authority in the 
language in which the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language: , which is: 

□ the language of a translation furnished for the purposes of the international search (under Rule 23.1 (b)). 

□ the language of publication of the international application (under Rule 48.3(b)). 

□ the language of a translation furnished for the purposes of international preliminary examination (under Rule 
55.2 and/or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the 
international preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in written form. 

□ filed together with the international application in computer readable form. 

□ furnished subsequently to this Authority in written form. 

□ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in 
the international application as filed has been furnished. 

□ The statement that the information recorded in computer readable form is identical to the written sequence 
listing has been furnished. 

4. The amendments have resulted in the cancellation of: 

□ the description, pages: 

□ the claims, Nos.: 



Form PCT/IPEA/409 (Boxes l-VIII, Sheet 1) (July 1998) 



INTERNATIONAL PRELIMINARY 

EXAMINATION REPORT International application No. PCT/G BOO/00242 



□ the drawings, sheets: 

5. □ This report has been established as if (some of) the amendments had not been made, since they have been 

considered to go beyond the disclosure as filed (Rule 70.2(c)): 

(Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this 
report.) 

6. Additional observations, if necessary: 



V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 

1. Statement 

Novelty (N) Yes: Claims 1-24 

No: Claims 

Inventive step (IS) Yes: Claims 1-24 

No: Claims 

Industrial applicability (IA) Yes: Claims 1-24 

No: Claims 



2. Citations and explanations 
see separate sheet 



VII. Certain defects in the international application 

The following defects in the form or contents of the international application have been noted: 
see separate sheet 



Form PCT/IPEA/409 (Boxes t-Vlll, Sheet 2) (July 1998) 



INTERNATIONAL PRELIMINARY International application No. PCT/G BOO/00242 
EXAMINATION REPORT - SEPARATE SHEET 



Concerning Box V 

The subject-matter of claims 1 to 24 meets the requirements set out in Articles 33 (2) to 
(4) PCT with respect to the documents cited in the International Search Report, for the 
reasons set out below. 

EP-A-0 542 377 Discloses the possibility of defining playback sequences for CD-ROM- 
XA type discs. These sequences may include linked lists comprising any combination of 
one or more audio segments and one or more (still) images. How the images and audio 
segments were captured is not disclosed. 
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image and one audio clip are stored in a linked group (see data structure 470 defined in 
Figure 9 and utilised in Figure 10). It is also suggested that the image may be a plurality 
of images such as time lapse, burst or movie images. However, this means that there is 
one audio clip and several images in a linked group (470). The other possibilities 
disclosed in the present application, viz. one image and multiple audio clips, or multiple 
images and multiple audio clips in one linked group are not functionalities which are 
hinted at in this document, nor is the desirability of such functionality hinted at. In fact, 
the bulk of the disclosure relates to the provision of thumbnail images in the data 
structure (470) to speed up index image display in edit mode. 

Note that the skilled person would not consider combining the above documents, in 
particular as the EP document does not even hint at the use of a digital camera (with 
sound capability). 

US-A-4 924 303 Discloses an interactive audio clip and video frame retrieval system. 
The audio and video data are distributed in the VBI of a TV signal, and may be 
"captured" and stored by a subscriber. There is no suggestion of the use of a digital 
camera (with sound capability). 
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(54) Title: DIGITAL CAMERA WITH SOUND RECORDING 
(57) Abstract 

A digital camera (1) for capturing sound passages as well as still 
images has the following elements. It has a camera apparatus (21) for 
capturing still images. It also has a sound recording apparatus (101, 
102, 103, 106) for capturing sound passages. A memory (107) is used 
for retaining still images as still image files and sound passages as 
sound passage files. The user interface of the camera is adapted 
for selectively linking one or more still image files with one or 
more sound passage files to form a linked group, and for selectively 
unlinking one or more files from a linked group. This equivalent 
treatment of still images and sound passages allows effective use of 
the camera for recording ambient sound together with images, and for 
flexible construction of sound/image composites representing a user's 
experience in a rich and effective manner. 
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DIGITAL CAMERA WITH SOUND RECORDING 

The present invention relates to a digital camera adapted to allow recording of sound 
5 as well as still images. In particular, the invention relates to a digital camera 
arrangement designed to support the construction of different combinations of sound 
and images. 

It is known that the combination of a still image and sound is a valuable one for the 
10 user, not only where the sound is provided as annotation, but also where the sound is 

ambient at the time of recording the still image. Examples of systems adapted for 

recording both still images and sound together are disclosed in US Patents Nos. 

3439598, 4270853, 4270854, 4905029 and 5276472. A system for display and 

playback of composite sound and still image is set out in European Patent Application 
15 Publication No. 0827018. This combination of sound and still image is also 

obtainable from certain digital video cameras which have capacity to capture still 

images - an example is the Hitachi MP-EG1 A. 

A camera operating system which supports augmentation of still images with sound is 
20 Digita, provided by Flashpoint Technology, Inc. Digita provides a menu-driven 
interface, and also scripts which allow construction of composite items at the camera 
(such as a page written in HTML with camera-captured image inserts). 

In existing products, the sound capture is generally optimised for annotation, rather 
25 than for recording of ambient sound. Accordingly, the microphone is directed 
towards the user rather than to the imaged scene, the sound is recorded after the image 
is captured, and the combination of media is limited to one sound clip per image. 
Although capture of ambient sound is a known use of cameras with sound capture 
features, existing arrangements still provide a simple relationship of association 
30 between a single still image and a single piece of sound. 

It is desirable to improve the capture of sound together with still images to allow a 
rich combination of the two which will provide particular benefits to the user. 
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Summary of Invention 



Accordingly, the invention provides a digital camera comprising: camera apparatus 
for capturing still images; sound recording apparatus for capturing sound passages; a 
memory for retaining still images as still image files and sound passages as sound 
passage files; and a user interface adapted for selectively linking one or more still 
image files with one or more sound passage files to form a linked group, and for 
selectively unlinking one or more files from a linked group. 

Still image files and sound passage files may be files of different types, in which case 
selective linking between files may be recorded in an index file containing all linkage 
information. Alternatively, linked groups may have a specific multimedia filetype 
supporting still image files, sound passage files, and links therebetween: in this case, 
still images and sound passages may also themselves be provided in this filetype, as it 
possesses the inherent capacity to be either a still image file, a sound passage file, or 
both. 

By treating the images and sound as equivalent forms of data in this way, rather than 
merely considering a sound passage as an optional addition to a still image, richer 
possibilities for creation and editing of associated sound and image data become 
available. This equivalence makes selection of sound and image to be matched 
together straightforward. Advantageously, it also allows for creation of more 
complex objects, with a linked group containing any of one still image file with one or 
more sound passage files; one sound passage file with one or more still image files; or 
even a plurality of both sound passage files and still image files. This flexibility is 
particularly important where ambient sound is to be captured: for example, a desirable 
approach may be to record the sound, and during recording of the sound to capture a 
large number of still images, deciding at a later point which images to keep. With a 
system in which sound is provided through annotation of a single image, this desirable 
mode of operating is simply not possible. 



Advantageously, the user interface is adapted such that a user can select a first and 
second view of the image files, wherein in the first view a plurality of sound passage 
files, still image files or linked groups are displayed in an index format for browsing, 
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and wherein in the second view a single sound passage file, still image file or linked 
group is displayed for review and editing (including modification of any link). It is 
helpful if in this second view an additional sound passage file, still image file or 
linked group is also displayable, and the user interface is adapted to allow linkage 
between the displayed files to form a new linked group. Another desirable feature is 
for the sound passage files and still image files of a linked group to be viewable, 
selectively, either as part of the linked group or independently of the linked group. 
Another useful feature is for the user interface is adapted so that the sound passage 
files, the still image files, and the linked objects are all presentable to the user as a 
separate sequence, and that the user may select between these separate sequences for 
browsing. This collection of features allows effective browsing through a collection 
of captured data, and effective combination and review of the data collected. 

It is particularly useful, especially in the case of more complex linked groups, for the 
digital camera to be adapted such that a still image file may be associated with a 
particular point of time or period of time in the sound passage of a sound passage file. 
This is valuable if images and sound passages can be captured contemporaneously. In 
connection with this, it is desirable for there to be a means to mark a captured still 
image or a captured sound passage with the time of recording. This time stamping 
can either be absolute, or in the case of contemporaneous recording, relative (that is, 
an image is stamped with its time of recording relative to the start of the sound 
passage). Contemporaneous recordings of sound and still images may be provided 
automatically as linked groups. This structure allows valuable forms of image 
association to be created, such as an assemblage of still images appearing in a 
determined succession and spacing against a common passage of ambient sound. 

In a development of this approach, embodiments of a camera may include a further 
data recording device for recording of another data type, and wherein the memory 
device is adapted to record the other data type as a further data type file, and wherein 
the user interface is adapted for selectively linking one or more further data type files 
into a linked group, and for selectively unlinking one or more further data type files 
from a linked group. These further data types may be graphic annotation (for 
example, handwriting input) or video clips. 
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p ?y T jptir>n nf Fibres and Snecific Embodiment 

Specific embodiments of the invention are described below, by way of example, with 
reference to the accompanying drawings, in which: 

Figure 1 shows a system for the capture and reproduction of sound and still images 
according to an arrangement proposed by the present applicant; 

Figures 2a, 2b and 2c show front (image and sound capture) side, rear (user interface 
side) and top (capture button) views of a digital camera in accordance with an 
embodiment of the invention; 

Figures 3a to 3c show the user interface display of Figure 2 in editing mode, .and 
illustrate selection of different data elements; 

Figure 4 shows the user interface display of Figure 2 in editing mode, and shows an 
audiophoto consisting of a sound passage with three attached images; 

Figures 5a to 5c show the user interface display of Figure 2 at three different stages 
20 during the playing of the audiophoto of Figure 4; 

Figure 6 shows the user interface display of Figure 2 in editing mode showing an 
index image of captured data; and 

25 Figures 7a and 7b show the user interface display of Figure 2 in capture mode, and 
illustrate respectively a viewfinder display and an index display. 



15 



Basic components in a system for providing an image and related sound are shown in 
30 Figure 1. A camera 1 comprises a conventional digital camera mechanism for 
capturing a still image together with a digital sound recording device for recording a 
passage of sound. Sound recording means are conventional: a microphone 101, an 
automatic gain controller 102, an analog/digital converter 103 and a codec 106 - 
where playback is also available, as in the embodiment to be described below, a 
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loudspeaker 105 and a digital/analog converter 104 are also be provided. A memory 
107 for storage of audio and image information is also required. 

It should be noted that although the term "image" is used here, it does not refer merely 
5 to a pictorial image. It may also relate to, for example, text captured in an image 
format. Likewise, when the term "camera" is used, other devices for capturing a still 
image can be understood to fall within its scope in the context of the present 
invention. 

10 Whereas the process of image and sound capture may be entirely analog, in which 
case both an analog image recording medium (such as film) and an analog sound 
recording medium are provided at the camera, in the context of the present invention 
digital capture of both sound and image is used. Both image and sound are digitized 
on capture and stored in an appropriate storage medium (for example on a 
15 conventional flash memory or an Iomega "Clik" disk ("Clik" is a trade mark of 
Iomega Corporation)). In this event, the digital data can be provided to a digital 
processing means (such as personal computer 2). This is advantageous, as it allows 
for easy editing of both image and sound data. The image data can then be provided 
from the digital processing means as a printed image, and the passage of sound 
20 attached thereto in a digital electronic storage medium, such as a flash memory. This 
is best achieved by means of a printer 3. In printer 3, the image is printed in 
conventional manner and a sound reproduction device with the passage of sound 
recorded thereon is attached to the image. Advantageously, recording and attachment 
of sound is done in the printer itself, but this step may be carried out separately (for 
25 example, at the sound recording device 4, as is described further below). 
Alternatively, it is possible in some embodiments for the image data and optionally 
the sound data to be provided directly from the camera to the printer (for example by 
infra red data transfer using technologies currently known for infra red printing from 
personal computers - such as under the IrTranP standard): these arrangements may 
30 require the processor in camera 1 to be provided with a printer driver for the 
appropriate printer. 

The result is a tangible representation of the image with an electronic storage medium 
having the passage of sound recorded thereon attached to it: this is here termed an 
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"audiophoto". To play the passage of sound, sound reproduction device 4 is 
employed. This can be connected to the electronic storage medium to enable 
information stored in the electronic storage device to be transferred to the sound 
reproduction device, the sound reproduction device 4 containing means to convert the 
information received from the electronic storage device into the passage of sound. 
However, sound reproduction device 4 is adapted to be detachable or otherwise 
remote from the printed image when no connection between the electronic storage 
device and the sound reproduction device is required. 

Further details of these system elements are provided in copending European Patent 
Application No. 98305436.2, the contents of which are incorporated by reference 
herein to the extent permissible by law. A digital camera 1 which is an embodiment 
of the present invention will now be discussed below, with reference to Figures 2a to 
2c. 

Figure 2a shows the front (image capture) side of a digital camera 1 according to an 
embodiment of the present invention. In basic form, digital camera I is a 
conventional handheld camera. This camera is optimised for capture of ambient 
sound, rather than annotation of images, so in addition to a conventional digital 
camera apparatus for capturing still images shown here by lens 21, the front side of 
the camera 1 also has the microphone 101 of a sound recording apparatus (of the kind 
discussed with reference to Figure 1) for capturing sound passages. In a digital 
camera optimised for annotation, this microphone 101 would be placed on the rear 
(user interface) side of the digital camera 1 - alternatively, dual microphones, or 
means to reorient the microphone for capture from different directions at different 
times, may be provided. The captured images and sound are eventually stored (in 
formats and associations discussed below in the context of the user interface of the 
digital camera) on a memory for retaining still images as still image files and sound 
passages as sound passage files. This memory comprises a removable flash memory 
, card 107 held in a slot on the side of the camera, and released from the slot by a 
button release 22. 

Figure 2b shows the rear (user interface) side of the digital camera 1. The display 23 
is (preferably) an LCD or other flat screen display. The display can be used either for 
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image and sound capture (capture mode) or for image and sound editing (editing 
mode) - both capture mode and editing mode are discussed further below. The 
display 23 is shown in Figure 2b in editing mode, with an audiophoto image shown 
with three display elements: image element 231, sound element 232, and selection 
element 233, all of which will be addressed below in the discussion of editing mode. 
Switch 26 is used to. allow the user to toggle between capture mode and editing mode. 
Also provided on the user interface side of the digital camera is loudspeaker 102, so 
that audio playback is directed towards the user. 

The user interface controls 24 are provided on the user interface side of the digital 
camera. The user interface controls 24 are used to browse, organise and edit the 
captured images and sound, and in particular to allow selective linking of one or more 
still image files with one or more sound passage files to form a linked group, and for 
selectively unlinking one or more files from a linked group. The specific functions of 
15 different buttons will be described in greater detail in the discussion of the editing 
mode, but a brief summary is as follows: 

data type selection button 241 allows selection of image, sound or both as 

"active"; 

deletion button 242 allows deletion of the current active element; 
20 linking/unlinking button 243 allows selected elements to be linked or 

unlinked; 

marking button 244 allows a sound clip to be marked (for reasons discussed 
further below); 

forward button 245 allows a user to step forward throughout data while 
25 browsing, or to play a sound passage; 

backward button 246 allows a user to step backward throughout data while 
browsing; and 

stop button 247 allows a user to stop playing of a sound passage. 



30 



Figure 2c shows a top view of the camera 1. The top of the camera is where data 
capture controls 25 are located. Data capture controls 25 are essentially conventional: 
there is an image capture button 251 easily operable with the right hand to record an 
image, and a dual state sound capture start button 252 easily operable with the left 
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hand to begin recording of sound on a first button press to end recording of sound on a 
second button press. 

Editing mode will now be discussed with reference to Figures 3 to 6. 

Figure 3a shows an audiophoto in editing mode. The audiophoto consists of a still 
image 231 and a sound passage (provided as a graphical representation 232 of the 
sound). The sound passage is marked with a tag 31 to indicate the association of an 
image with it - the position of the tag 31 is indicative of the time along the sound 
stream represented by the graphical representation 232. The sound passage is also 
marked with its duration 32. Selection element 233 indicates in Figure 3a that the 
whole audiophoto is selected. The selection element can be toggled between the 
states of audiophoto/sound element/image element by pressing the data type selection 
button 241. 

When the audiophoto is selected as in Figure 3a, the options provided by the different 
user interface buttons are as follows. Forward button 245 causes the display to 
advance to, and play, the next audiophoto. Backward button 246 causes the display to 
move to, and play, the previous audiophoto. Stop button 247 causes the sound 
passage to stop playing. Linking/unlinking button 243 is for linking or unlinking the 
displayed elements: for an audiophoto, the display at any one time will be showing an 
image element 231 and a sound element 232 that are linked to each other. Pressing 
the linking/unlinking button 243 with an audiophoto selected will generally be used to 
enable the linkage between the image element 231 and the sound element 232 to be 
broken: the result could be separated image and sound, separated audiophoto and 
sound, separated audiophoto and image, or even two separated audiophotos (if both 
the image and the sound have additional attachments) - it can also be used to modify 
the link. Both deletion and modification of links are discussed below with reference 
to Figure 3b. The deletion button 242 in this mode causes deletion of the selected 
audiophoto (confirmation by pressing a different designated key would be required in 
an effective practical interface), and marker button 244 is used to mark the sound 
passage 232, as is discussed further below with reference to Figure 3b. 
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If instead the sound passage is selected, as in Figure 3b, then the meaning of the 
buttons will be different in some cases. The forward button 245 and the backward 
button 246 now cause display, and play, of the next and previous sound passage 
respectively. No change is made to the displayed image. Consequently, selection of 
sound passages only can be used to allow cycling through sound clips to find a clip 
for attachment to the presently displayed image. Stop button 247 is used to stop 
playback of a sound clip. Deletion button 242 is used to delete the selected sound 
passage. Linking/unlinking button 243 is now used essentially for creating or 
removing links between the selected sound and the currently displayed image. 
Pressing the button 243 causes a link to be established - or if a link already exists 
between the sound passage and the image, for that link to be removed. This link may 
represent a relationship in time between the sound passage and the image: in 
particular, for which parts of the sound passage the image is to appear. There may be 
a simple one to one relationship with no element of time (in which case, the image 
will simply be associated with the beginning of the sound passage), but there may be a 
sequence of images appearing successively at different times during the sound 
passage. To support this, the user interface provides means on creation of a link to 
position that link within the sound passage. When the link is created, it will appear 
provisionally at the start of the sound passage (unless capture was contemporaneous - 
see below), and can be moved forward or backward along the sound passage by use of 
markers, as described below. In a variant of this approach, if capture of images and 
sound are time stamped (either absolutely, or if contemporaneously captured images 
and sound are time stamped relative to each other), the initial position of a link 
between contemporaneously recorded sound and images can be at the actual point of 
capture of the image within the sound record. 

Use of marking within the sound record can be used to improve construction of the 
relationship between images and sound. While a sound passage is playing, pressing 
of the marker button 244 places a marker (not shown in Figure 3a or Figure 3b, but 
visible as a visual symbol in sound record 232) in the sound record at a time 
corresponding to the point in the sound record at which the marker was pressed. 
When the sound record is not playing, pressing of the marker button 244 places the 
sound clip into a state in which the markers can be used for editing. Initially, a 
passage in the sound record up to the point of the first marker is selected. Pressing 
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forward button 245 advances selection to the first marker itself, and further pressing 
forward button 245 advances selection to the passage of sound between the first and 
second markers. Pressing the button again advances to the second marker, and 
successive presses alternate between selection of markers and intervening passages 
until the end of the sound clip is reached, at which point the process starts again from 
the beginning of the clip. In this way, either a marker itself, or the sound passage 
between two markers, can be selected. By use of the backward button 246 it is 
possible to cycle through the markers and intervening passages in the opposite 
direction. In this way, a marker or intervening passage may be selected and either 
deleted with delete button 242 (and so edit the sound clip itself) or used to set up a 
link with an image (by using linking/unlinking button 243), the link thus establishing 
either when the image will display within the resulting audiophoto (if the link is with 
a marker) or over what period of time within the resulting audiophoto that the image 
will display (if the link is with an interval). 

The alternative to selecting the audiophoto or selecting the sound passage is to select 
the image, and this is shown in Figure 3c. The forward button 245 and the backward 
button 246 now cause display of the next and previous images respectively. No 
change is made to the displayed sound passage. Consequently, selection of images 
only can be used to allow cycling through all the images for attachment to the 
presently displayed sound passage. Deletion button 242 is used to delete the selected 
image. Linking/unlinking button 243 is now used essentially for linking the selected 
image with the currently displayed sound, using principles discussed above in relation 
to Figure 3b. 

Although not shown here, practical embodiments will require a confirmation step for 
significant events, such as deletion of an audiophoto, image or sound clip. 
Advantageously, an "undelete" function will also be provided to allow deletions to be 
recovered to whatever extent is practical. 

A highly advantageous feature of the embodiment of the present invention is that it 
supports audiophotos that are more than a simple association of a sound passage with 
an image: in particular, an audiophoto may comprise a sound passage with one or 
more images, each of which has a relationship in time with the sound passage, or 
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alternatively a collection of more than one sound passage with one or more images. 
An example of an audiophoto with a single sound passage but a plurality of images is 
shown in Figures 4 and 5. Figure 4 shows the initial view of such an audiophoto, with 
the audiophoto selected but the sound not playing. There are three images 42, 43, 44 
associated with the sound passage represented by trace 41, each of which are 
represented in the trace 41 by a discrete clip icon 45, 46, 47. The clip icons are 
located in positions corresponding to the point in the sound passage at which they are 
intended to appear. The change in the display as the sound passage plays is shown in 
Figures 5a to 5c. 

Clip icon 45 is located at the start of the sound passage, so image 42 is displayed from 
the beginning of the sound passage. A visual representation for the amount of the 
sound passage so far played is provided by a background bar 48 behind trace 41. In 
Figure 5a, it can be seen that the sound passage is part played, but that the second clip 
icon 46 has not yet been reached. In consequence, the first image 42 is being 
displayed. When the second clip icon 46 position is reached, then the second image 
43 will be displayed. Figure 5b shows display of the second image 43 at a point 
during playback of the sound passage after the second clip icon 46 has been reached 
but before the third clip icon 47 has been reached. After the third clip icon 47 has 
been reached, the third image 44 is displayed. In this case, each of the images is 
associated with a point in time (indicating the beginning of display of that image) 
rather than a period of time (as would be the case if the image was linked, through use 
of markers, with a part of a sound clip). There is no basic difference between the two 
situations - however, linking with passages of time can be used to provide dark 
intervals between pictures, or overlaps. 

It is desirable for there to be a view of the various available audiophotos and elements 
which renders it easier to compare different elements, or to move between elements to 
be edited. Accordingly, the editing mode may also provide an index view for viewing 
multiple elements, or collections of elements. Such an index view 61 is illustrated in 
Figure 6. This index view includes all elements that are not part of an existing 
audiophoto, together with existing audiophotos. A selection frame 62 is provided to 
allow selection for viewing and playing (as in Figure 3a to 3c) of a particular element 
or audiophoto - such selection can be achieved through use of forward button 245 and 
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backward button 246. Delete button 242 is used to delete the elements within 
selection frame 62. An appropriate button combination is used to switch between 
single audiophoto view and index view. When in index view, transition to single 
audiophoto view is achieved by pressing data type selection button 241 while stop 
button 247 is being pressed. The reverse transition from single audiophoto view to 
index view can be achieved by the same button combination. When switching from 
index view to single audiophoto view, the display 23 will change to contain the 
element or elements within the selection frame 62 in single audiophoto view. 

It is possible from the index view to determine whether a particular entry is an image, 
a sound passage, a simple audiophoto or a more complex audiophoto (not a simple 1:1 
relation of image and sound). Images 64 and sound passages 65 are readily 
identifiable as they contain no sound trace and no image respectively. Simple 
audiophotos 63 can be seen to contain one sound trace and one image. More complex 
arrangements, such as a single image with two sound clips 66 or a single sound clip 
with multiple images 67, are again clearly identifiable as instead of providing a 
representation indicative of the start (or any other point) of the sound passage, an icon 
is provided for every element. For example the image 66 with two sound clips has a 
single image shown but two sound traces shown: however, the image 67 with a single 
sound clip but three images shows all three images in the image view 67, rather than 
just a single image associated with a particular point in time. The image view is thus 
an effective way to determine quickly the composition (at least the image 
composition) of existing audiophotos. 

Different forms of image view are possible. An alternative could be to show all 
images, or all sound samples, or all audiophotos, and to toggle- between these 
alternatives with the data type selection button 241. Another alternative would be to 
provide separate index displays of sound passages and images. Images and sound 
samples could be displayed according to their time of capture (particularly appropriate 
if some form of absolute time stamping is available). 

By pressing button 26, the camera is switched between editing mode as discussed 
above, and capture mode. The initial image in capture mode will be as for a 
conventional digital camera: a simple image indicating what would be recorded if at 
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that point image capture button 251 were to be pressed. If the image capture button 
251 is pressed at this point, then a simple image results (like images 64 in Figure 6). 
However, the position changes when recording of a sound passage is initiated by 
pressing sound capture button 252. This causes the display to appear as shown in 
Figure 7a, with a sound bar 71 showing the build up of the sound passage (the length 
of sound so far captured, together with an indication that recording is in progress, are 
also provided) and a viewfinder image 72 below. If sound recording stop button 253 
is pressed before image capture button 251 is pressed, then a pure sound passage will 
be recorded, like sound passage 65 in Figure 6. However, if image capture button 251 
is pressed while sound recording is building up, then a contemporaneous sound and 
image capture occurs. Advantageously, this links the captured image and the sound 
passage 71 directly, with the link registered for initial purposes at the point during the 
sound passage at which the image capture occurred (of course, the time relationship in 
the link can be changed at a later point through editing). 

If an audiophoto is built up in this way, it is desirable for an alternative view to be 
available to illustrate all the elements of the audiophoto. Accordingly, if the data type 
selection button 241 is pressed during capture, then the image display is toggled 
between the viewfinder image 72 shown in Figure 7a to a composite of captured 
20 image thumbnails 73, each marked with their time of recording relative to the start of 
the sound passage. The representation of the sound passage 71 is marked in this view 
with indicators 74 showing the time of recording for the different images. 

In addition to pure capture and pure review, it is desirable and possible to support an 
25 intermediate arrangement in which new audio or image data can be inserted 
selectively into the existing set, rather than simply added to the end. 

Insertion of such an additional image or sound segment is here carried out from the 
review mode, with the appropriate context (audiophoto, sound clip, or image) for the 
30 insertion being selected in the single audiophoto view on display 23 . 

When the insertion context is a sound clip, the user presses image capture button 251 
to attach a new photo to the clip, or presses sound capture button 252 twice to capture 
a new sound passage which is appended to the end of the existing sound clip. If the 
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user wishes to insert a new sound segment at an appropriate point during the existing 
sound passage, this can be done by advancing to and marking with marker button 244 
the desired insertion point, and in that state capturing the new sound clip. 

When the insertion context is a still image without sound, the user can simply press 
the image capture button 251 to add a new image after the existing image, or use the 
sound capture button 252 to record an audio segment which is then attached to the 
selected image. 

When the insertion context is an existing audiophoto, the user can press the image 
capture button 251 to attach a new image to the sound clip, creating or extending a 
series of photos for the same clip. The point in the sound passage at which the new 
image is introduced can be automatically assigned, or manually positioned by use of 
the marking button 244 as previously described. Alternatively, the sound capture 
button 252 can be used to record a further sound segment, either to append to the 
audioclip, or to position within the audioclip by use of the marking button 244 as 
previously described. 

After capture, the following information is stored in the memory 107: images (stored 
20 as discrete files), sound passages (stored as discrete files), and timestamping and 
(where relevant) linking information. There are two possible approaches for this, 
either of which can be used in satisfactory embodiments. The first is for the image 
and sound to be stored in their own separate data types, with all linking, timing and 
selection information stored in a separate index file. This solution is probably the 
25 simplest to implement, but is not convenient for selective transfer of information from 
memory 107 to another device. The second approach is for there to be a multimedia 
filetype for audiophotos, supporting image, sound, timing and linking information. If 
such a filetype is used, then this filetype can be used for all information captured 
(including images alone, and sounds alone) so all files processed by the camera can be 
30 of the same filetype. This solution allows each resultant audiophoto to be freestanding 
as a single file, without reference to an index, rendering selective transfers from 
memory 107 straightforward. This information can then be transmitted in any of the 
paths shown in Figure 1: to a personal computer 2 (an effective route for complex 
audiophotos - which could then be transmitted, for example, by electronic mail), 
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directly to a printer 3 or to a reader appliance 4. For simple images, sound passages, 
or audiophotos, sending directly to a printer or reader appliance is essentially as 
described in copending European Patent Application No. 98305436.2, but differences 
are required if there are multiple images in an audiophoto. For example, a complex 
audiophoto sent directly to a printer could be rendered with the visual element shown 
in a composite form as shown in Figure 6. The reader appliance 4 could also be 
replaced with a composite reader/viewer appliance (not shown) capable of displaying 
images and playing back sound to play back an audiophoto as captured or devised at 
the camera I: the functions to be provided need only be those offered by the user 
interface of the camera 1 (possibly retaining the browsing features but not the editing 
features), though with preferably sufficiently high quality display and sound playback 
to provide a satisfactory experience to the user. 

In addition to still images and sound, it is possible for other data types to be captured 
at a digital camera. A digital camera can typically be adapted to capture and store 
short bursts of video without difficulty: such video bursts may be provided in the 
same manner as sound passages as an augmentation of a still image. Similarly, means 
to provide a graphic input at the user interface (for example, a pen-based digitiser 
interface and means for recognising handwriting or a custom alphabet such as 
Graffiti) may be used to provide graphic input to provide a similar augmentation. 
Such augmentations may be treated in essentially the same way as sound passages, 
and may be used as further (or conceivably alternative) elements in a linked group. 

As the skilled man will appreciate, many variations upon the features indicated above 
can be made in devising alternative embodiments of the invention. For example, the 
structural arrangement of the camera can be changed considerably - display 23 may be 
used only as a user interface display, with a separate viewfinder being placed 
elsewhere on the camera, or a different arrangement and type of controls can be 
employed for the user interface. 
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CLAIMS 



1 . A digital camera comprising: 

camera apparatus for capturing still images; 

sound recording apparatus for capturing sound passages; 

a memory for retaining still images as still image files and sound 

passages as sound passage files; and 

a user interface adapted for selectively linking one or more still image 
files with one or more sound passage files to form a linked group, and 
for selectively unlinking one or more files from a linked group. 

2. A digital camera as claimed in claim 1, wherein a linked group may consist of 
15 one still image file with one or more sound passage files. 

3. A digital camera as claimed in claim 1, wherein a linked group may consist of 
one sound passage file with one or more still image files. 

20 4. A digital camera as claimed in any of claims 1 to 3, wherein the user interface 
is adapted such that a user can select a first and second view of the image files, 
wherein in the first view a plurality of sound passage files, still image files or 
linked groups are displayed in an index format for review, and wherein in the 
second view an individual sound passage file, still image file or linked group 

25 is displayed for editing. 

5. A digital camera as claimed in claim 4, wherein in the second view an 
additional sound passage file, still image file or linked group is also 
displayable, wherein the user interface is adapted to allow linkage between the 
30 displayed files to form a new linked group. 



6. 



A digital camera as claimed in claim 4 or claim 5, wherein in the second view 
a further sound passage or still image may be. captured and linked 
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automatically to the individual sound passage file, still image file, or linked 
group displayed. 

A digital camera as claimed in any preceding claim, wherein the digital 
camera is adapted such that a still image file may be associated with a 
particular point of time or period of time in the sound passage of a sound 
passage file. 

A digital camera as claimed in any preceding claim, wherein the digital 
camera is adapted such that the camera apparatus may capture one or more 
still images at the same time that the sound recording apparatus is capturing a 
sound passage. 

A digital camera as claimed in claim 8, wherein any still images captured at 
the same time as a sound passage are provided initially as part of a linked 
group with the sound passage. 

A digital camera as claimed in any preceding claim, further comprising a time 
recording means to mark a captured still image or a captured sound passage 
with the time of recording. 

A digital camera as claimed in claim 10 where dependent on claim 8, wherein 
the time recording means is adapted to mark the time of capture of a captured 
still image relative to a contemporaneously recorded sound passage. 

A digital camera as claimed in any preceding claim, wherein the user interface 
is adapted for selective deletion by a user of a still image file, a sound passage 
file or part of a sound passage file, or a linked group. 

A digital camera as claimed in any preceding claim, wherein the user interface 
is adapted such that sound passage files and still image files of a linked group 
are viewable, selectively, either as part of the linked group or independently of 
the linked group. 
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A digital camera as claimed in claim 13, wherein the user interface is adapted 
so that the sound passage files, the still image files, and the linked objects are 
all presentable to the user as a separate sequence, and that the user may select 
between these separate sequences for browsing. 

A digital camera as claimed in any preceding claim, wherein the memory 
retains linked groups as separate sound passage files, still image files, and one 
or more index files containing linking information. 

A digital camera as claimed in any preceding claim, wherein the memory 
retains linked groups as multimedia files, containing sound passage 
information, still image information, and linking information. 

A digital camera as claimed in claim 16, wherein the sound passage files and 
the still image files are both multimedia files. 

A digital camera as claimed in any preceding claim, further comprising a 
further data recording device for recording of another data type, and wherein 
the memory device is adapted to record the other data type as a further data 
type file, and wherein the user interface is adapted for selectively linking one 
or more further data type files into a linked group, and for selectively 
unlinking one or more further data type files from a linked group. 

A digital camera as claimed in claim 18, wherein the further data type is 
handwriting data, and the further data recording device is a device for 
recording handwriting in a digital representation. 

A digital camera as claimed in claim 18, wherein the further data type is a 
video clip, and the further data recording device is a device for recording 
video clips. 

A digital camera as claimed in any preceding claim, wherein the sound 
recording apparatus and the camera apparatus are arranged for data capture in 
a substantially common direction. 
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22. A digital camera as claimed in any preceding claim, wherein the memory is a 
flash memory device. 

23. A digital camera as claimed in any preceding claim, wherein the digital 
camera is adapted to be handheld by a user for still image and sound passage 
capture. 

24. A program for operating the user interface of a digital camera comprising 
camera apparatus for capturing still images and sound recording apparatus for 
capturing sound passages, wherein the program comprises means for linking 
one or more still images with one or more sound passages to form a linked 
group, and means for unlinking one or more elements from a linked group. 
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Figure 3c 
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